Repository navigation
Feat: generate llms.txt and markdown pages for AI agents - #1470
Merged
Merged
Conversation
- Add docusaurus-plugin-llms to emit llms.txt indexes and a markdown copy of every docs page next to its HTML (<page-url>.md) - Mirror the site's version routing: the root index covers the latest release, with nested indexes for the CLI reference, next and older versions under /docs/<version>/llms.txt - Keep every index under the ~100K character budget agents can read in one fetch; the postbuild script fails the build if one grows past it - Point agents at the index from every page: a hidden note in the HTML (src/theme/Root.js) and a blockquote atop each generated markdown file Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in>
anoop2811
requested review from
FogDong,
briankane,
jguionnet and
roguepikachu
as code owners
November 12, 2026 19:00
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 5 files
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
- The self-contained Dockerfile invokes docusaurus build directly, which skips yarn's postbuild hook; chain the postprocessor so every build path produces the same output (CI already uses yarn build) Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in>
jguionnet
approved these changes
Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Makes the documentation consumable by AI agents: generates an llms.txt index and a markdown copy of every docs page at build time.
docusaurus-plugin-llms(new dev-time plugin) emitsllms.txt,llms-full.txt, and a.mdtwin next to every page's HTML, so agents can fetch<page-url>.mdinstead of parsing themed HTML. Version output mirrors the site's routing: the rootllms.txtindexes the latest release served at/docs/, whilenextand older versions get their own indexes under/docs/<version>/llms.txt.scripts/llms-postprocess.js(runs via thepostbuildhook, so CI and Netlify-styleyarn buildboth trigger it) restructures the indexes into the llmstxt nested layout: CLI man-pages move to/docs/cli/llms.txtand other versions hang off/docs/versions/llms.txt. This keeps every index in the 77–91K character range — under the ~100K budget agents can ingest in one read — and the script fails the build if an index ever crosses it. It also prepends an index pointer to every generated markdown file.src/theme/Root.jsadds a visually hidden, server-rendered note on each page telling agents where the index and markdown variants live.Measured with afdocs (the scanner behind CLOMonitor's agent readiness section) against a local production build: overall 94/100, with content discoverability at 92 and index coverage at 100%. The one category that still fails —
Accept: text/markdowncontent negotiation — is a hosting limitation, not something the repo can fix.I have:
sidebar.jsif adding a new page.yarn startto ensure the changes has taken effect.Special notes for your reviewer
No docs pages or sidebar entries change — all new output is generated into
build/(verified with fullyarn buildruns rather thanyarn start, since the plugin and postbuild hook only run on production builds). Build time grows by roughly 1–2 minutes for the markdown generation across versions. The commit is DCO signed off.Summary by cubic
Makes the KubeVela docs consumable by AI agents: builds an
llms.txtindex and a markdown copy of every page atdocsbuild time so agents can fetchhttps://kubevela.io/llms.txtor any page's.mdtwin instead of parsing styled HTML. Measured 94/100 on the CLOMonitor agent readiness scan (up from no index at all), with content discoverability 92 and index coverage 100%.Build changes
docusaurus-plugin-llmsdependency and configures it to emit per-version indexes matching the site's routing (/docs/for the latest release,/docs/<version>/fornextand older versions).scripts/llms-postprocess.jsas apostbuildhook that restructures the plugin output into the llmstxt nested layout: CLI man-pages move under<version>/cli/llms.txt, per-version indexes under/docs/versions/llms.txt, and every index stays under the ~100K character agent ingestion budget (the script fails the build if one exceeds it).Dockerfilebuild, which runsdocusaurus builddirectly and would otherwise skip yarn'spostbuildhook.llms.txton every generated markdown page and adds a visually hidden note to every HTML page viasrc/theme/Root.js.No docs pages or sidebar entries change; all output is generated into
build/. Build time grows by roughly 1–2 minutes for markdown generation.Written for commit 2c722c3. Summary will update on new commits.